ACT Working Paper 2016 (05) 


ACT Reporting Category Interpretation Guide 

Version 1.0 


Sonya Powers, PhD 
Dongmei Li, PhD 
Hongwook Suh, PhD 
Deborah J. Harris, PhD 

October, 2016 


ACT Working Paper Series 

ACT working papers document preliminary research. 
The papers are intended to promote discussion and 
feedback before formal publication. The research does 
not necessarily reflect the views of ACT. 






Sonya Powers is a senior psychometrician at ACT specializing in test equating, score 
comparability, and testing of special populations. 

Dongmei Li is a principal psychometrician at ACT specializing in test equating, scaling, 
and growth modeling. 

Hongwook Suh is a psychometrician at ACT specializing in equating, linking, and test 
response time analysis. 

Deborah J. Harris is vice president of psychometrics at ACT specializing in equating, 
linking, and the comparability of reported scores. 


The authors would like to thank all the other ACT team members who helped with the 
document, including Yi-Fang Wu, Lu Wang, Shalini Kapoor, Yong He, and Benjamin 
Andrews. 


ACT Reporting Category Score Interpretation Guide 
Version 1.0 


Abstract 

ACT reporting categories and ACT Readiness Ranges are new features added to the ACT score reports 
starting in fall 2016. For each reporting category, the number correct score, the maximum points 
possible, the percent correct, and the ACT Readiness Range, along with an indicator of whether the 
reporting category score falls within the Readiness Range are provided. In the format of guestions and 
answers, this document first gives an introduction of the reporting categories in terms of what they are, 
how they are reported, and how they are related to the previous subscores. Then, based on results from 
12 ACT forms administered to randomly eguivalent groups of about 3,000 examinees in each, 
information is provided on the technical characteristics of the reporting category scores, including score 
reliability, standard error of measurement, and classification consistency of the ACT Readiness Ranges. 
This document is intended to facilitate the interpretation of reporting category scores and the ACT 
Readiness Ranges. 


1 1 p a g e 



What are the ACT reporting categories? 


The ACT reporting categories are new scores reported on the ACT® score reports beginning in fall 2016. 
There are three reporting categories each for English, reading, and science, and eight reporting 
categories for mathematics. These scores break overall test performance into smaller categories that 
can be used to identify components of the subject area that are relative strengths or weaknesses for 
each test taker. 

Tables 1 through 4 provide a description of the ACT reporting categories for each subject, including the 
target range of items within each reporting category. The number of items within a reporting category 
may fluctuate slightly across forms though the total number of items within a subject is the same across 
forms. Older ACT test forms may deviate slightly from these target ranges, but new forms are built to 
match the target ranges as closely as possible. 


Table 1. ACT English Reporting Categories 


Reporting Category Labels and Descriptions 

Target Ranges 

Number 

of Items 

Percentage 
of Test 

Production of Writing 

• Demonstrate an understanding of, and control over, the rhetorical 
aspects of texts 

• Identify purposes of parts of texts 

• Determine whether a text or parts of text have met the goal 

• Evaluate the relevance of material in terms of a text's focus 

• Use various strategies to ensure that a text is logically organized, 
flows smoothly, and has an effective introduction and conclusion 

22-24 

29-32% 

Knowledge of Language 

• Demonstrate effective language use through ensuring precision and 
concision in word choice and maintaining consistency in style and 
tone 

11-13 

15-17% 

Conventions of Standard English 

• Apply understanding of relationships between and among clauses, 
placement of modifiers, and shifts in sentence construction 

• Edit text to conform to Standard English usage 

• Edit text to conform to Standard English punctuation 

39-41 

52-55% 

TOTAL 

75 

100% 
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Table 2. ACT Mathematics Reporting Categories 


Reporting Category Labels and Descriptions 

Target Ranges 

Number 

of Items 

Percentage 
of Test 

Preparing for Higher Math 

34-36 

57-60% 

Number & Quantity 

• Demonstrate knowledge of real and complex number systems 

• Understand and reason with numerical quantities in many forms, 
including integer and rational exponents, vectors, and matrices 

4-6 

7-10% 

Algebra 

• Solve, graph, and model multiple types of expressions 

• Employ different kinds of equations, for example, linear, polynomial, 
radical, and exponential 

• Find solutions to systems of equations, even when represented by 
simple matrices, and apply their knowledge to applications 

7-9 

12-15% 

Functions 

• Understand function definition, notation, representation, and 
application for linear, radical, piecewise, polynomial, logarithmic, 
and other functions 

• Manipulate and translate functions 

• Apply important features of graphs 

7-9 

12-15% 

Geometry 

• Define and apply knowledge of shapes and solids, such as 
congruence and similarity relationships or surface area and volume 
measurement 

• Understand composition of objects 

• Solve for missing values in triangles, circles, and other figures, 
including using trigonometric ratios and equations of conic sections 

7-9 

12-15% 

Statistics & Probability 

• Describe center and spread of distributions 

• Apply and analyze data collection methods 

• Understand and model relationships in bivariate data 

• Calculate probabilities including the related sample spaces 

5-7 

8-12% 

Integrating Essential Skills 

• Use essential skills (i.e., concepts typically learned before 8 th grade, for 
example, rates, percentages, proportional relationships, area, surface 
area, volume, average, median, etc.) to... 
o Solve problems of increasing complexity 
o Combine skills in a longer chain of steps 
o Apply skills in more varied contexts 
o Understand more connections 

o Become more fluent 

24-26 

40-43% 

Modeling* 

• Produce, interpret, understand, evaluate, and improve models 

> 16 

>27% 

TOTAL 

60 

100% 


*Each modeling item is also included in another reporting category. 
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Table 3. ACT Reading Reporting Categories 


Reporting Category Labels and Descriptions 

Target Ranges 

Number 

of Items 

Percentage 
of Test 

Key Ideas and Details 

• Determine central ideas and themes 

• Summarize information and ideas accurately 

• Make logical inferences 

• Understand sequential, comparative, and cause-effect relationships 

22-24 

55-60% 

Craft and Structure 

• Determine the meaning of words and phrases 

• Analyze an author's word choice rhetorically 

• Analyze text structure 

• Understand authorial purpose and perspective 

• Analyze characters' points of view 

• Differentiate between various perspectives and sources of information 

10-12 

25-30% 

Integration of Knowledge and Ideas 

• Understand authors' claims 

• Differentiate facts and opinions 

• Use evidence to make connections between different texts that are 
related by topic 

• Analyze how authors construct arguments 

• Evaluate reasoning and evidence from various sources 

6-7 

15-18% 

TOTAL 

40 

100% 


Table 4. ACT Science Reporting Categories 



Target Ranges 

Reporting Category Labels and Descriptions 

Number 

of Items 

Percentage 
of Test 

Interpretation of Data 

• Manipulate and analyze scientific data presented in tables, graphs, and 
diagrams (e.g., recognize trends in data, translate tabular data into 
graphs, interpolate and extrapolate, and reason mathematically) 

18-22 

45-55% 

Scientific Investigation 

• Understand experimental tools, procedures, and design (e.g., identify 
variables and controls) 

• Compare, extend, and modify experiments (e.g., predict the results of 
additional trials) 

8-12 

20-30% 

Evaluation of Models, Inferences, and Experimental Results 

• Judge the validity of scientific information 

• Formulate conclusions and predictions based on scientific information 
(e.g., determine which explanation for a scientific phenomenon is 
supported by new findings) 

10-14 

25-35% 

TOTAL 

40 

100% 
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How do the ACT reporting categories relate to the previous ACT subscores? 

The ACT reporting categories replace the previous ACT subscores for English, reading, and mathematics, 
and also add reporting categories for science. The new reporting categories, which are reported 
beginning in fall 2016, are designed to provide information better aligned with college and career 
readiness standards. 

Some of the ACT reporting categories measure skills similar to the previous subscores, leading to high 
correlations between the new ACT reporting category scores and the previous ACT subscores. For 
example, the previous Usage/Mechanics subscore for English and the new Conventions of Standard 
English reporting category scores are very highly related. Likewise, the Intermediate Algebra/Coordinate 
Geometry subscore is very highly related to the Preparing for Higher Math reporting category. However, 
for other ACT reporting categories, the scores are based on different skill sets and are only moderately 
correlated with previous subscores. As an example, the previous reading subscores were content-based 
(e.g., Social Studies/Sciences and Arts/Literature) and are only moderately correlated with the reading 
Integration of Knowledge and Ideas reporting category. 

What information is reported for the ACT reporting categories? 

Underneath the overall subject test scale score on the 
student score report is a list of the reporting 
categories and scores. For each reporting category, 
the total number of items, the number of correct 
responses, the percent of correct responses, and the 
ACT Readiness Range, along with an indicator of 
whether the reporting category score falls within the 
Readiness Range (©) are provided. 

What is the ACT Readiness Range and how is it determined for the reporting categories? 

ACT student data are used to create a predictive relationship between the ACT College Readiness 
Benchmark score on the overall test and each of the test's reporting categories. So, for example, a 
Readiness Range is developed for each of the three English reporting categories. For the first reporting 
category, student scores on the overall English test and scores on the Production of Writing reporting 
category are used to estimate the predictive relationship between the two scores. This relationship is 
then used to identify the minimum percent correct value for the reporting category that corresponds to 
the benchmark score on the overall English test. Students with percent correct values at or above the 
minimum percent correct value obtained during this process are considered to be within the ACT 
Readiness Range. The same process is repeated to determine Readiness Ranges for the Knowledge of 
Language and Conventions of Standard English reporting categories. 

The Readiness Range is intended to provide more meaningful comparisons of student performance 
across reporting categories. A comparison of percent correct values may lead to incorrect conclusions 
about student strengths and weaknesses because the difficulty of the items within each reporting 
category is likely to differ. However, even the Readiness Ranges must be interpreted cautiously because 
some of the reporting categories are based on a small number of items. In fact, when the Readiness 
Ranges were retroactively applied to a sample of 12 ACT forms administered in 2015 to randomly 
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equivalent groups of about 3,000 examinees in each, the percent of students within the ACT Readiness 
Range varied somewhat dramatically across forms for certain reporting categories. Table 5 provides the 
minimum, maximum, and the maximum difference in the percent of students within the ACT Readiness 
Ranges across the 12 forms for each reporting category. These differences cannot be attributed to group 
differences as the administration of these forms was designed to achieve very similar samples of 
students across forms. These fluctuations in the percent of students within the Readiness Range is 
largely a function of how many items are within a particular reporting category. For example, the 
percent of students within the ACT Readiness Range varied less than 5% across the 12 forms for 
Conventions of Standard English which has about 40 items, but varied more than 20% across forms for 
Number & Quantity which has about 5 items. More information on the stability of reporting category 
scores is provided in the Technical Characteristics section below. 

Table 5. Percent of Students within the ACT Readiness Range across Twelve 2015 Forms Administered to 
Randomly Equivalent Groups of Examinees 


Test/Reporting Categories 

Median 

# of Items 

Percent of Students within the 

ACT Readiness Range 

Minimum Maximum Difference 

English 

Production of Writing 

23 

62.70 

68.91 

6.21 

Knowledge of Language 

12 

60.51 

73.67 

13.16 

Conventions of Standard English 

40 

64.75 

69.56 

4.81 

Mathematics 

Preparing for Higher Math 

35 

40.35 

48.67 

8.32 

Number & Quantity 

5 

29.88 

52.64 

22.76 

Algebra 

8 

38.66 

59.57 

20.91 

Functions 

8 

35.84 

56.12 

20.28 

Geometry 

8 

37.36 

57.08 

19.72 

Statistics & Probability 

6 

35.17 

58.95 

23.78 

Integrating Essential Skills 

25 

41.84 

49.90 

8.06 

Modeling 

24 

38.89 

51.92 

13.03 

Reading 

Key Ideas & Details 

23 

47.42 

56.01 

8.59 

Craft & Structure 

11 

45.76 

58.71 

12.95 

Integration of Knowledge & Ideas 

6 

40.15 

67.57 

27.42 

Science 

Interpretation of Data 

18 

38.87 

52.23 

13.36 

Scientific Investigation 

10 

33.38 

57.45 

24.07 

Evaluation of Models, Inferences, & 

Experimental Results 

12 

36.92 

51.13 

14.21 


How can the ACT reporting category scores be used? 

For each reporting category, the total number of items, the number of correct responses, the percent of 
correct responses, and the ACT Readiness Range are provided. Because the items in different reporting 
categories may vary in difficulty, comparing the number of points earned and the percent correct values 
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across the reporting categories is problematic. However, the ACT Readiness Range provides information 
with a consistent interpretation, that is, whether the student's performance in a particular reporting 
category is above or below the typical performance of students who meet the ACT College Readiness 
Benchmark on the subject test. This information provides an at-a-glance score profile that can be used 
to identify student strengths and areas for improvement. Students can use this information to focus 
their remediation efforts so they can better prepare themselves for college success. 

Figure 1 provides an example score report for each of the four ACT subject areas and their 
corresponding reporting categories. Using science as an example, this hypothetical student received an 
overall score of 18 which is below the College Readiness Benchmark score of 23. This score indicates 
that overall the student would benefit from additional science coursework or remediation so that he or 
she is adequately prepared for college. However, a closer look at the student's reporting category score 
profile reveals that the student is struggling more with some concepts than with others. For skills related 
to Interpretation of Data, the student only answered 56% of the items correctly which falls below the 
ACT Readiness Range for that reporting category. However, on the next reporting category, Scientific 
Investigation, the student was able to answer 70% of the items correctly, which was a score high enough 
to fall within the ACT Readiness Range. On the last reporting category, Evaluation of Models, Inferences, 
and Experimental Results, the student answered only 36% of the items correctly and will need to 
improve their performance in order to be prepared for success in college on these types of science skills. 
These results suggest that this student would benefit most from additional practice, coursework, or 
remediation in the science skills related to the first and third reporting categories, though more caution 
is needed when interpreting reporting category scores based on small numbers of items. 


Detailed Results 
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Figure 1. Example ACT Student Score Report 

By breaking down the overall score into smaller component skills, it is also possible to learn more about 
individual students. For example, three students might all have an overall science score of 18. However, 
the score profile for each student may reveal a different pattern of strengths and weaknesses. Figure 2 
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provides an example of three students each having one reporting category score within the ACT 
Readiness Range and two reporting category scores below the ACT Readiness Ranges. Student 1, 
represented by the green line, has a profile similar to the one described in Figure 1, with a relative 
strength in the second reporting category (RC2), and relative weaknesses in the first (RC1) and third 
reporting categories (RC3). Student 2, represented by the blue line, shows a relative strength in RC1 but 
relative weaknesses in RC2 and RC3. Student 3, represented by the orange line, demonstrates the best 
performance in RC3 relative to the other two reporting categories. This example serves to illustrate the 
usefulness of the reporting category scores. Even though these three students may have the same 
overall science score of 18, the subskills that should be the focus for improvement are different for each 
student. Without the reporting category scores, students lack information about which skills or concepts 
they need to focus on to improve their preparation for college success. 



Figure 2. Example ACT Science Score Profiles 


What are the technical characteristics of the ACT reporting categories? 

Reliability is a measure of the consistency of student scores. Reliability values range from 0 to 1 with 0 
representing completely unreliable scores and 1 representing perfectly reliable scores. 

The standard error of measurement (SEM) is another measure that quantifies score consistency, but it is 
quantified in terms of score points. For example, an SEM of approximately 2 for the number of correct 
responses on a reporting category indicates that students' number of correct responses is likely to vary 
within a range of plus or minus 2 if a different set of items on this reporting category were administered. 
In the case of the SEM, for a given reporting category, smaller numbers are better because they reflect a 
higher degree of consistency and confidence in the reported score. 

Reliability and SEM for the ACT reporting categories are provided in Table 6. These values were 
calculated using 12 forms administered in 2015 to randomly equivalent groups of approximately 3,000 
students per form. 
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Table 6. Reliability and SEM for the ACT Reporting Category Scores 



Median 

# of 

Items 

Reliability 


SEM 

Number Correct 

Score 

Percent 

SEM 

Correct Score 

Test/Reporting Categories 

Median 

Min 

Max 

Median 

Min 

Max 

Median 

Min 

Max 

English 

Production of Writing 

23 

0.81 

0.77 

0.82 

2.07 

1.99 

2.12 

9.02 

8.64 

9.20 

Knowledge of Language 

12 

0.67 

0.61 

0.72 

1.46 

1.36 

1.52 

12.20 

11.35 

12.66 

Conventions of Standard English 

40 

0.86 

0.84 

0.88 

2.67 

2.58 

2.74 

6.67 

6.44 

6.86 

Mathematics 

Preparing for Higher Math 

35 

0.84 

0.80 

0.86 

2.51 

2.47 

2.56 

7.17 

7.05 

7.31 

Number & Quantity 

5 

0.35 

0.26 

0.54 

0.95 

0.89 

1.00 

19.02 

17.71 

20.05 

Algebra 

8 

0.54 

0.49 

0.65 

1.19 

1.13 

1.23 

14.82 

14.15 

15.39 

Functions 

8 

0.59 

0.45 

0.65 

1.20 

1.15 

1.24 

14.99 

14.35 

15.56 

Geometry 

8 

0.55 

0.48 

0.60 

1.22 

1.13 

1.26 

15.20 

14.13 

15.76 

Statistics & Probability 

6 

0.42 

0.34 

0.51 

1.06 

1.01 

1.09 

17.62 

16.80 

18.10 

Integrating Essential Skills 

25 

0.81 

0.77 

0.84 

2.11 

2.07 

2.17 

8.43 

8.26 

8.66 

Modeling 

24 

0.79 

0.71 

0.84 

2.07 

1.73 

2.32 

8.73 

7.75 

10.79 

Reading 

Key Ideas & Details 

23 

0.78 

0.73 

0.80 

2.10 

2.02 

2.21 

8.99 

8.21 

9.23 

Craft & Structure 

11 

0.60 

0.54 

0.65 

1.45 

1.32 

1.54 

13.17 

12.85 

14.72 

Integration of Knowledge & Ideas 

6 

0.43 

0.34 

0.55 

1.09 

0.82 

1.14 

18.63 

17.01 

21.14 

Science 

Interpretation of Data 

18 

0.72 

0.60 

0.76 

1.77 

1.51 

1.88 

10.01 

9.32 

10.86 

Scientific Investigation 

10 

0.59 

0.47 

0.69 

1.39 

1.17 

1.71 

13.28 

11.26 

16.74 

Evaluation of Models, Inferences, & 

Experimental Results 

12 

0.61 

0.45 

0.74 

1.55 

1.28 

1.69 

12.95 

10.56 

16.00 


For some of the reporting categories, particularly those with very few items, the reliability is much lower 
than is typically seen on an overall subject test. Less stability in the reporting category scores is the 
trade-off of having scores based on fewer items. However, the reporting category scores are not 
intended for high stakes decisions, but instead are intended to guide instruction and help identify 
students' strengths and weaknesses. 

Caution is needed when comparing the SEM of number correct scores across reporting categories with 
different numbers of items. Although the SEM values based on the number correct score scale do not 
appear larger for the reporting categories with a smaller number of items, an SEM of approximately 1 
for scores based on 5-8 items represents more uncertainty compared to an SEM of approximately 2 for 
23-25 items. Another way to compare the SEM values is to use the percent correct score scale. A 
comparison of the percent correct SEM values shows the expected pattern where reporting categories 
with lower numbers of items have higher SEM values. For example, the Number & Quantity reporting 
category, which contains approximately 5 items, has a median SEM of 0.95 on the number correct scale, 
but a median SEM of 19.02 on the percent correct scale. If a student received a score of 4 out of 5 (80%), 
a range of plus or minus 19% would put the likely score for this student anywhere from 61% to 99%. This 
amount of variability around the student's scores makes it difficult to be confident about whether the 
student falls within or below the ACT Readiness Range. 
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Classification consistency provides an indication of the stability of student classifications. In the case of 
the ACT reporting categories, the classification of interest is whether or not the student falls within the 
ACT Readiness Range. Conceptually, classification consistency values indicate the proportion of students 
who would receive the same classification if tested a second time. Using Table 7 as an example, if 100 
students take the ACT twice, then classification consistency is the proportion of students who are either 
below the Readiness Range for a particular reporting category based on both tests, or within the 
Readiness Range for a particular reporting category based on both tests. Consistent classifications are 
highlighted in yellow. Twenty students in this example were below the Readiness Range on both the 
initial test and the re-test. Fifty students in this example were within the Readiness Range on both the 
initial test and the re-test. Therefore, 70 out of 100 students were classified consistently. 


Table 7. Classification Consistency Example 



C 

assification on Re-Test 


Classification on 

Initial Test 


Below 

Readiness Range 

Within 

Readiness Range 

Total 

Below 

Readiness Range 

20 

20 

40 

Within 

Readiness Range 

10 

50 

60 


Total 

30 

70 

100 


However, since many students only take the test once, we only have one classification. Models have 
been developed to estimate classification consistency even with a single classification per student. One 
such model, developed by Livingston and Lewis (1995), 1 was used to calculate classification consistency 
for the ACT reporting categories. Classification consistency values can range from 0 to 1 with values 
closer to 1 indicating more stable classifications. Higher stability in student classifications is desired so 
that score users can be confident in the classification they received (i.e., below the ACT Readiness Range 
or within the ACT Readiness Range). 

Classification consistency of 12 forms administered in 2015 is reported in Table 8. Median classification 
consistency values ranged from 0.61 to 0.85 across reporting categories. As expected, lower classification 
consistency values were observed for reporting categories with fewer items. 

The results in Tables 6 and 8 highlight the need for caution when interpreting student scores and 
classifications based on small numbers of items. Reporting category scores are reported for remediation 
and targeted instruction, not for college admissions or placement decisions. 


1 Livingston, S. A., and Lewis, C. (1995). Estimating the consistency and accuracy of classifications based on test scores. Journal 
of Educational Measurement, 32(2), 179-197. 
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Table 8. Classification Consistency Values for the ACT Readiness Ranges 



Median 

# of Items 

Classification Consistency 

Test/Reporting Categories 

Median 

Min 

Max 

English 

Production of Writing 

23 

0.82 

0.79 

0.83 

Knowledge of Language 

12 

0.76 

0.75 

0.82 

Conventions of Standard English 

40 

0.85 

0.83 

0.86 

Mathematics 

Preparing for Higher Math 

35 

0.83 

0.81 

0.85 

Number & Quantity 

5 

0.61 

0.58 

0.72 

Algebra 

8 

0.68 

0.66 

0.75 

Functions 

8 

0.72 

0.67 

0.75 

Geometry 

8 

0.69 

0.66 

0.73 

Statistics & Probability 

6 

0.66 

0.61 

0.70 

Integrating Essential Skills 

25 

0.81 

0.79 

0.84 

Modeling 

24 

0.81 

0.76 

0.83 

Reading 

Key Ideas & Details 

23 

0.80 

0.78 

0.82 

Craft & Structure 

11 

0.73 

0.69 

0.75 

Integration of Knowledge & Ideas 

6 

0.65 

0.60 

0.70 

Science 

Interpretation of Data 

18 

0.77 

0.73 

0.79 

Scientific Investigation 

10 

0.71 

0.68 

0.76 

Evaluation of Models, Inferences, & 

Experimental Results 

12 

0.73 

0.68 

0.80 
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